00100	During the past several years our  vision  effort  has  shifted  from
00200	simple scenes of polyhedra to complex objects in richer context.  Not
00300	all problems with towers of polyhedra have been solved, but there are
00400	a  host  of  reasons  for moving on.  Manipulation has also proceeded
00500	from manipulation of blocks to assembly operations for  many  of  the
00600	same  reasons.     Minsky and Papert [Minsky 1972] have described the
00700	central  issues  and  the  intellectual  position  of   robotics   in
00800	artificial intelligence. We refer to that discussion as background to
00900	our motivation to direct research to real world scenes  and  objects.
01000	An  important  motivation is to develop applications for manipulation
01100	and visual feedback capabilities which have been developed here.  The
01200	time  is  ready  for  the  first transfer of robotics technology from
01300	laboratory  to  prototype  assembly  stations.  We  discuss  robotics
01400	applications more fully in another section.
01500	
01600	Another  major  motivation is to face new issues; systems have come a
01700	few  steps  toward  heterarchical  systems  from  heirarchy;  from  a
01800	succession  of homogeneous passes with compounding errors to a system
01900	structure where high and low levels interact closely.  One aspect  of
02000	such a system is the use of many functionally independent techniques;
02100	this has meant including  experimentation  with  color  and  texture,
02200	depth  information, motion parallax and image prediction.  The choice
02300	of problems as paradigms is central to the way  in  which  perceptual
02400	systems  evolve.      A  great deal can be done with simple scenes of
02500	polyhedra using a single image  in  one  color,  ignoring  all  other
02600	perceptual abilities.  Since most perceptual problems can be resolved
02700	within  a  single  image,  one   approach   is   to   exhaust   those
02800	possibilities.      Important  areas  are missed; for example, visual
02900	feedback is clumsy without depth information.   For our plans, visual
03000	feedback for manipulation is primary.
03100	
03200	There  are  other  ways  in  which  the  tower  of blocks paradigm is
03300	inadequate  for  our  purposes.      Low  level  feature  description
03400	operators   are   tailored  for  this  case.    Adequate  region  and
03500	edge-oriented descriptors must be evolved.     With  polyhedra,  only
03600	simple  preliminary  grouping  mechanisms have been programmed.  More
03700	intermediate level descriptors must be implemented.
03800	
03900	Although real world scenes are much more  complicated,  they  may  be
04000	simpler  to  perceive in that they have a much richer context.  There
04100	is only very general knowledge available in the  usual  blocks  task;
04200	usually  color,  size,  and  support are unspecified.  In an assembly
04300	task, sizes are known, color is  used  to  code,  and  parts  fit  in
04400	special  ways.     Often  recognition can be based on obvious special
04500	features.  These tasks are good exercise in developing  goal-oriented
04600	visual  systems.  Although development and utilization of intelligent
04700	robots will progress gradually, the information sciences will  impact
04800	industry  enormously  over  a shorter term. An immediate effect is to
04900	streamline information flow in the manufacturing  process.     Simple
05000	systems  can keep track of delayed effects of design decisions, lower
05100	cost of preparation and  transmission,  and  increase  accessibility.
05200	Computer    controlled    scheduling,    resource   allocation,   and
05300	transportation routing are important to most of industrial production
05400	[Anderson  1972].   We see a major contribution from techniques being
05500	pioneered in artificial  intelligence  laboratories,  and  that  such
05600	research  will  have  payoff  throughout  the  entire  development of
05700	intelligent systems. Enormous efficiency  increases  will  come  from
05800	analysis  of  assembly  and  inspection  in  terms of some "primitive
05900	operations".   In the past, computer cost and lack of  sophistication
06000	has  limited  the  ability  for  such  analysis, and the slow pace of
06100	automation has not pressed  hard  for  analysis.    Now  it  will  be
06200	possible   to   find   common  fabrication  and  assembly  operations
06300	throughout a large company, to discover  commonality  in  design  and
06400	components.  The success of that analysis depends upon representation
06500	of fabrication operations and shape.   Representation  is  a  current
06600	concern  in  robotics  labs  and  may  have  early applications. This
06700	orientation to the manufacturing system is already beginning to  have
06800	effect  with  the  "group  technology"  approach,  which  attempts to
06900	describe parts and operations, and group them together in one part of
07000	the   factory.     Until  the  establishment  of  such  analysis  and
07100	commonality  of  operations,  automating  will  be  made  much   more
07200	difficult.
07300	
07400	Considerations are similar for programming automation devices such as
07500	the assembly station we describe.  Even if programming is  difficult,
07600	there  will  be  considerable  applications  for  dexterous  assembly
07700	systems. For example, for the military, it is preferable to stockpile
07800	programs than to stockpile parts or systems for contingency. In batch
07900	production, tapes can be saved to cut stockpiling.   But  to  achieve
08000	full  effect  on  industry, robot systems must be easy to program for
08100	new tasks.  Our current work is concerned  with  strategy  generation
08200	and  automating  visual  feedback.    Time-sharing  systems  will  be
08300	necessary  to  perform  scheduling,  resource   allocation,   routing
08400	operations,  etc.   Thus there is a natural division of labor between
08500	the time-sharing system and small machines. The strategy  generation,
08600	automating   feedback,  and  interactive  system  for  computer-aided
08700	mechanical design expect to reside in the central system.
08800	
08900	A major part of our effort is directed toward the  representation  of
09000	complex  shapes,  for  use  in vision and strategies.  While it might
09100	seem that this effort has only distant applications, for the  reasons
09200	discussed   above,   early   applications   will  be  for  convenient
09300	man/machine   interaction   for   programming    robots    and    for
09400	systematization.   We can expect that in general, much of the work in
09500	symbolic representation of the real world  will  have  early  payoff.
09600	Present  non-intelligent  robots  are programmed by specifying points
09700	along a path  to  be  repeated.    With  computer  control,  optional
09800	trajectories  can  be  executed in case of contingency.  The engineer
09900	must then specify what normal function is, how to monitor it, and how
10000	to  recognize recoverable situations.   To do so, he can specify this
10100	symbolically in interaction with a  program  in  which  he  specifies
10200	normal  and abnormal conditions, shape of parts (usually some variant
10300	of standard parts), the mating of parts for assembly,  and  schematic
10400	descriptions   of  operations  for  insertion  and  inspection.   The
10500	significance of representations in general is to aid these processes.
10600	Our representation provides a natural way of speaking about the shape
10700	of objects, in terms of a structured description of  part/whole  with
10800	primitives  which  resemble many fabrication operations.   The better
10900	the representation, the more it provides intuition for programs as to
11000	the intent of operations and the function of parts.
11100	
11200	A  representation  aids communication in both directions.  We contend
11300	that our representation is important for  computer  graphics.    This
11400	allows  display  of  the  machine's  concepts to the user, and allows
11500	greater use of predictive mechanisms based on internal models.  Input
11600	could  be made visually with a depth measuring system such as we have
11700	been using for the last two years.  Since the operation would be done
11800	on  a central facility, the computer would be adequate for this task.
11900	We expect that the representation will  be  of  considerable  use  in
12000	monocular  vision  also.   We hope to substantiate these predictions.
12100	Another application is  automating  design  of  numerically  machined
12200	parts.    That  will be simplified to the extent that the machine and
12300	the user speak the same language of subparts and their  descriptions.
12400	Benefits  would  come  from  ease  of  programming  and  interaction,
12500	variable  tolerances  for  critical  parts  and   better   monitoring
12600	operations.
12700	
12800		THE PILOT ASSEMBLY STATION
12900	
13000	We  seek  other  funds  to  partially support application of robotics
13100	techniques to industrial assembly and inspection.   We will implement
13200	over  the  next  few years a prototype assembly station and a support
13300	system for programming the assembly station for its tasks.
13400	
13500	The assembly station is assumed to be normally connected to  a  large
13600	time-shared computer facility. Not only does this appear to be a good
13700	design for our problem, but it is also necessary  for  an  integrated
13800	programmed automation facility.  The only emphasized statement in the
13900	ARPA study report [Anderson  1972]  is:     "The  maximum  impact  of
14000	computers   in  manufacturing  will  come  from  complete,  real-time
14100	computer cognizance and  control  of  all  processes  and  resources,
14200	allowing the precise scheduling and allocation of these resources."
14300	
14400	Each  assembly  station will consist of a small digital computer, one
14500	or more manipulators and cameras. Manipulators and  cameras  will  be
14600	combined  as  modules  in  systems  to suit a given task, and will be
14700	digitally controlled by the station computer.
14800	
14900	The program in the station computer will be flexible as to the number
15000	and  position of manipulators and cameras. It will control input from
15100	the cameras, provide for some pre-processing of the picture  and  the
15200	application  of  simple  tests to the visual data.     In the case of
15300	manipulators it will provide for simultaneous servo control. It  will
15400	also  process  other  inputs,  such  as  from  touch  sensors  on the
15500	manipulators, from photocell interrupt devices, limit switches,  etc.
15600	Both  analog  and binary outputs will be available for the control of
15700	other devices.
15800	
15900	Each station computer will have access to a task  program  specifying
16000	the actions and tests it should perform to carry out its task.   This
16100	task program will be compiled by the central computer.    Compilation
16200	will  consist of the analysis of predicted scenes in order to specify
16300	simple tests that the station computer can carry out, the calculation
16400	of  collision free trajectories for the manipulators, the sequence of
16500	operations and simple contingency plans. etc.  We  are  planning  for
16600	tasks   whose  normal  execution  does  not  require  detailed  scene
16700	analysis.  If scene analysis becomes necessary (e.g. a dropped  part)
16800	we  envision the station sending an image to the central computer for
16900	processing.
17000	
17100	The central computer will perform overall control of  the  individual
17200	assembly  stations  and the relative rates of production.   If at any
17300	station a planned test fails, for which a contingency plan  does  not
17400	exist,  or  any  unexpected  sensor signals are received, the central
17500	computer will also be interrupted and appropriate actions taken.
17600	
17700	Most of the hardware required  for  the  assembly  station  has  been
17800	developed  for  the  laboratory.    Manipulators  and cameras will be
17900	discussed in detail below. The major additional problem is to  refine
18000	the   existing   designs   to  meet  the  demands  of  an  industrial
18100	environment.
18200	
18300	The most difficult part of the assembly station, and the heart of the
18400	of the proposal is a demonstration of the feasibility of the station.
18500	We plan to do this by developing programs which actually assemble and
18600	inspect a variety of fairly complex manufactured items.
18700	
18800	A  typical  example is our current project, assembly of an automobile
18900	water pump. The major  parts  are  to  be  located,  the  screws  are
19000	supplied by a feeder, as is standard.  Locating the body and cover of
19100	the pump require a gross localization, then a  fine  localization  of
19200	screw  holes.      Touch  provides  a  final accuracy and independent
19300	coordination of the hand.   The  placing  of  alignment  pins  within
19400	screw  holes  could  be monitored visually; although in this case,the
19500	previous actions provide adequate accuracy.   Placing  of  the  cover
19600	over  alignment  pins  can  be  carried out without vision.  Improved
19700	force  sensing  now  under  development  will  be  useful  for   that
19800	operation.
19900	
20000	As  we move to more difficult tasks, the sophistication required goes
20100	up quite rapidly.  The  current  manipulation  routines  seem  to  be
20200	expandable  to cover anticipated problems, especially since we intend
20300	to employ special purpose manipulators (tools) for some tasks.    The
20400	main part of our effort will be vision for assembly; an assembly task
20500	will be performed using vision, touch, and two hands.     The  vision
20600	effort  will  be  integrated with other abilities, and while no great
20700	generality will be stressed, new modules will  be  incorporated  into
20800	the  system,  and a set of techniques will evolve.  These will depend
20900	heavily on techniques similar to  those  of  our  recent  studies  in
21000	visual  feedback  and hand/eye coordination [Gill 1972]. The internal
21100	model of the world will be  used  to  predict  visual  appearance  of
21200	selected  areas  of  the  scene, and a set of verification procedures
21300	will confirm or contradict the predictions.  Many of  the  operations
21400	will  be  connected  with  supplying  feedback  to  manipulation, for
21500	alignment and insertion. These visual procedures will have models for
21600	the parts and approximate locations.
21700	
21800	
21900	
22000	
22100	This and other visual tasks connected with assembly will initially be
22300	performed as special case problems.    In  addition,  to  make  these
22400	techniques   available,   we  will  pursue  development  of  strategy
22500	programs.  This step is essential if intelligent assembly devices are
22600	to  be conveniently programmed for complex tasks. Within the division
22700	of labor between large and small machines, this facility must  reside
22800	in  the  central  time-sharing  facility which has large memory, file
22900	storage, and problem-solving languages. The system must have a  model
23000	of  the  parts  for  assembly,and  a  sequence  of  instructions  for
23100	assembly. It will be necessary to have a representation of shape  and
23200	models for usual assembly operations such as insertion and alignment,
23300	and the ability to solve simple problems  in  connection  with  these
23400	operations,  all  this  to  enable  convenient communication with the
23500	user. When these facilities exist, the addition of new  features  and
23600	operations  can be incorporated smoothly, so that users can call on a
23700	growing library of planning modules.  The result of a planning effort
23800	is  a  cycle  of  machine  planning and man-machine interaction which
23900	results in a plan  network  for  the  mini  computer  which  controls
24000	individual machines.
24100	
24200		RESEARCH IN VISION
24300	
24400	Our  past work on visual feedback [Gill 1972] in stacking objects and
24500	inserting objects  in  holes  is  directly  relevant  to  vision  for
24600	assembly.   The  system provides feedback for picking up, then at the
24700	final position, with the hand still holding the block  and  in  view,
24800	predicts  the  appearance  of edges and corners within a small window
24900	around the faces to be aligned, and  uses  a  corner/edge  finder  to
25000	determine  alignment  error. Incremental motion capability of the arm
25100	is  used  for  error  correction.          That   work   incorporates
25200	self-calibration  to  coordinate  hand and eye coordinate systems and
25300	maintain reliability (by maintaining  calibration),  as  well  as  to
25400	limit   search   by  accurately  predicting  expected  features.  Two
25500	well-developed systems exist to aid future work in  visual  feedback.
25600	GEOMED  [Baumgart 1972a] is a geometric editor which allows the input
25700	of  geometric  models  and  manipulation  of  these  models.   OCCULT
25800	[Baumgart  1972b]  is a hidden line elimination program which gives a
25900	symbolic output, valuable for vision, in a few seconds.
26000	
26100	We have a set of visual  operations  which  provide  confirmation  of
26200	predictions  and  direct  interaction  with the visual image.     For
26300	outdoor scenes, color is an important basis for region  forming,  and
26400	for  preliminary scene organization. Several preliminary color region
26500	analysis programs have been developed [Binford,  unpublished,  Bajcsy
26600	1972].    These will be extended as described below. An edge operator
26700	[Hueckel 1969,1972] and a region-based corner/edge finder [Gill 1972]
26800	both  operate  on small windows of the scene, as directed by a higher
26900	level.     An edge verifier  is  used  to  confirm  or  reject  edges
27000	predicted  from  context [Tenenbaum 1970].   Color identification has
27100	long been demonstrated.   Recently, a program  has  been  implemented
27200	based  on  Land's Retinex model of color constancy in vision.    This
27300	involves taking intensity measurements with the camera  through  red,
27400	green,  and  blue filter, and setting up a calibration scheme for the
27500	Goethe color triangle in the absence of any external  color  standard
27600	(white).   This is achieved by normalizing the intensities of regions
27700	within the scene through each filter, and defining a  color  triangle
27800	derived  from  this  ordering  data,  with the center of the triangle
27900	designated as white. The regions may then be assigne color  names  in
28000	the  normal  way.   The program as it stands can cope reasonably well
28100	with table top scenes under varied lighting  conditions  -  sunlight,
28200	quartz-halogen  and flourescent, and therefore shows the same kind of
28300	color constancy as do humans.
28400	
28500	We have directed part of our work  to  analysis  of  outdoor  scenes.
28600	Outdoor  scenes were analyzed with a combination of color and texture
28700	region analysis  [Bajcsy  1972];  depth  cues  were  taken  from  the
28800	gradient of texture.  A symbolic representation of texture and of the
28900	world were adopted; descriptors of texture elements were obtained  as
29000	analytic  expressions  from  the  Fourier  spectrum.   A considerable
29100	improvement could be made by combining these techniques with  several
29200	techniques which obtain depth information from correlation of images.
29300	Motion parallax [Nevatia draft] has been used with  a  succession  of
29400	images.   The  successive  images  differ little from one another and
29500	allow local  correlation  without  ambiguity.     In  a  sense,  this
29600	combines the advantages of narrow and wide angle stereo, since stereo
29700	ambiguity is avoided, while large baselines  can  be  accumulated  by
29800	tracking  regions  over  several  images. Another system incorporates
29900	several facilities in wide angle stereo [Quam 1972]; local statistics
30000	are  used  to  limit  search for correspondence of small areas in two
30100	images.    An interesting feature is  self-orientation:    given  two
30200	cameras whose orientation is unknow, the system uses guided search to
30300	establish  a  few  correspondences,  then  determines  the   relative
30400	orientation   of   the   two   cameras.   Such  self-orientation  and
30500	self-calibration  facilities  have  considerable   significance   for
30600	industrial  systems  which can be maintained calibrated.    These are
30700	complementary depth sensing techniques; motion parallax can  be  used
30800	for  planning  paths and characterizing large scale terrain features.
30900	Two camera stereo can be used for  local  manipulation  and  disaster
31000	avoidance.   Visual  feedback  with stereo vision is potentially more
31100	powerful than without.    We have a program  which  has  demonstrated
31200	stereo visual feedback [Binford, unpublished].
31300	
31400	A  thesis  by  Agin  [Agin  1972]  showed  considerable  progress  in
31500	representation and description of complex objects.  He implemented  a
31600	laser  ranging  device [Binford 1970] which was operational two years
31700	ago.    Using depth data and a representation of shape [Binford 1971]
31800	which  we  have developed, we have been able to describe the parts of
31900	objects  like  a  doll,  snake,  and  glove  [Agin  1972].        The
32000	representation is based on segmenting objects into parts connected at
32100	joints.    Parts  are  described  by  a  volume  representation  with
32200	arbitrary  cross  section defined along a space curve.  The data were
32300	analyzed by techniques suggested by the representation.   Preliminary
32400	segmentation  was made into parts suggested by some primitives of the
32500	representation, smoothly varying,  locally  cylindrical  parts.    An
32600	iterative  procedure  defined  an  approximation  to  the axis of the
32700	segment, fit  circular  cross  sections  normal  to  the  axis,  then
32800	obtained  an  estimate  of  a global cross section function which was
32900	then used to extend the axis and cross section as far as consistent.
33000	
33100	A study of visual and depth ranging techniques has been  carried  out
33200	which  provides  a background for choosing sensors and interface, and
33300	making cost estimates [Binford 1972,Earnest 1967].    An  interesting
33400	result  is  that  the  cost of a laser ranging system such as ours is
33500	less than $1k given a tv system. We have maintained a calibrated hand
33600	and  vision system for several years [Sobel 1970], and have developed
33700	self-calibration  techniques  with  one  and  two  eyes   [Yakimovsky
33800	unpublished, Hannah (in [Quam 1972]), Gill 1972].
33900	
34000	A  system  for  understanding  scenes of blocks from single views has
34100	been implemented [Falk 1970]; it segments  objects,  matches  against
34200	internal  models,  predicts  appearance  from  its  assumptions,  and
34300	searches for verification and contradiction based  on  the  predicted
34400	image.    Support and spatial relations are also obtained.  There has
34500	been past work on programs  to  obtain  line  drawings  as  input  to
34600	recognition  systems.  An earlier system was based on limited context
34700	of expecting continuation of edges at vertices and predicting missing
34800	edges  in certain expected configurations [Grape 1970]. A more recent
34900	approach has been to use models within the process of generating line
35000	drawings,  so  that  the  most  reasonable next guess can be made and
35100	context used intrinsically in the formation of line drawings.
35200	
35300	VISUAL DEVELOPMENT FOR ASSEMBLY
35400	
35500	In a later section, we describe assembly of a pump carried  out  with
35600	the  manipulator  using  touch,  without vision.  We are implementing
35700	vision programs to aid the assembly. In  the  pump  assembly,  visual
35800	tasks  consist  of  gross  location  of  large  parts, then using the
35900	information to predict location of images of holes, and monitoring of
36000	insertion  into holes.   These tasks are typical of the simple visual
36100	operations which we will implement or adapt  for  assembly.  We  will
36200	greatly  expand our efforts with visual feedback.  At present, visual
36300	feedback techniques are limited to polyhera.  We will  improve  image
36400	prediction techniques to predict images of objects with curved edges.
36500	We will develop  a  simple  region  analysis  program  which  assumes
36600	adequate contrast to locate the curved edges predicted within a small
36700	window.   Color will be used when necessary.      We  will  implement
36800	two-dimensional  matching  of  boundary curves with predicted curves.
36900	These  will  be  combined  with  our  existing  edge   operator   and
37000	corner/line  operator.     We  will  maintain a calibrated system and
37100	improve calibration updating so that reliability is maintained  at  a
37200	high level.
37300	
37400	VISION RESEARCH
37500	
37600	We  will improve the laser hardware so that it can be used widely and
37700	extend our work with representation of complex  objects  using  depth
37800	data.     We  are able to make complete descriptions of objects which
37900	consist of a single primitive segment, such as a torus, a snake or  a
38000	cone.     The  programs can segment dolls, toy horses, in intuitively
38100	natural ways [Agin 1972]. It is necessary to improve our segmentation
38200	process  in  order to proceed to joining primitive segments at joints
38300	and make complete descriptions of complex objects.     When  we  have
38400	these  descriptions,  the computer will have symbolic internal models
38500	of objects.   No recognition has yet been done, although we are in  a
38600	position  to  recognize  those  objects which have only one primitive
38700	segment, for which we can make complete descriptions. For recognition
38800	of  more complex objects, once we have complete descriptions, we will
38900	encode the basic properties of the  representation  within  a  visual
39000	memory   scheme   which   allows   access   of  hypothesized  similar
39100	three-dimensional shapes, with  associated  surface  information,  to
39200	compare more closely with extensive verification.  We intend to apply
39300	large parts of our programs for description and recognition of curved
39400	objects  using  only  monocular intensity information. This is a much
39500	harder problem, but within our representation,  shares  much  of  the
39600	same  code,  and  relies  on  the  same  conceptual  basis  of  local
39700	three-dimensional models.
39800	
39900	
40000	We  will  continue  work  on  color region analysis which is based on
40100	proximity rather than contiguity.  This  was  the  plan  outlined  in
40200	previous proposals, and has shown necessary by our work with textured
40300	scenes [Bajcsy 1972].   Another effort combines region analysis  with
40400	a  decision  theoretic approach based on a world model. We anticipate
40500	that these modules will aid the integrated system to function in more
40600	complex  environments than have been attempted in the past.  There is
40700	a growing  effort  on  navigation  of  vehicles  in  outdoor  scenes.
40800	While this is not directly relevant to the industrial environment, it
40900	does reflect the growing involvement in  complex  visual  tasks,  and
41000	will lead to further development of an integrated visual system.
41100	
41200	A  strategy  project  is  underway  dealing with the world of blocks.
41300	The project begins from the idea that robot vision cannot be  studied
41400	except  as a part of intelligent behavior in general.  Visual ability
41500	is FOR something,  and  without  reasonable  expectations  about  the
41600	world,  the  task of seeing is bound to be extremely difficult.    We
41700	plan to implement a system which can  do  simple  manipulation  tasks
41800	("pick  up  the  red  block",  "put  all the cubes in the box") in an
41900	environment of plane-faced objects, and operating in a dialogue mode.
42000	The  system  will have a good but imperfect world model all the time,
42100	and the essence of its ability will be the constant use of this model
42200	to  understand  situations  and  in particular to recover from errors
42300	("Is that block no longer there because the arm  knocked  it  off?").
42400	The  main  point  of  the  project is to find a good organization for
42500	knowledge about the simple world in question:  we  expect  that  when
42600	this  is  done  the  complex  subroutines  of current vision programs
42700	(linefinders etc) will be found to be unnecessarily complex  in  that
42800	they embody in an inefficient way at a low level, knowledge which our
42900	system will contain at a higher level. In part, this will be done  by
43000	expanding  the facilities available.  A stereo vision module is being
43100	programmed, using line  drawings  and  integrating  stereo  with  the
43200	segmentation and recognition processes.
43300	
43400	We  are  at  work  on  a  program which exploits new features of SAIL
43500	[Feldman  1972]  for  dealing   with   multi-process   tasks   in   a
43600	problem-solving environment. The aim is to construct a driver program
43700	to  accomplish  a  simple  assembly  task  under  somewhat  uncertain
43800	conditions.  Making use of the new language features, one can arrange
43900	for such a program to cope with errors in the execution  of  tasks  -
44000	e.g. to recover from dropping things, breaking things, and further to
44100	try several different but possibly cooperative methods for solving  a
44200	problem.   It  is intended that this program will have its own vision
44300	procedures  specially  adapted  for  finding  tools,  e.g.  wrenches,
44400	screwdrivers, etc., and parts, e.g. nuts and bolts. An interpreter is
44500	being developed to allow flexible description  of  the  task  by  the
44600	user.
44700	
44800	MANIPULATOR DEVELOPMENT
44900	
45000	Manipulator   research  began  at  Stanford  Artificial  Intelligence
45100	Laboratory  in  1965  with  the  acquisition  and  programming  of  a
45200	prototype  Rancho Orthotic Arm.  A later version of this arm was used
45300	in vision directed block stacking programs.
45400	
45500	Our present arm, the result of a continuing design project,  replaced
45600	the  Rancho Arm in 1970. This arm like its predecessor was controlled
45700	by a digital servo program running under the time  sharing  computer.
45800	The new arm was used in the "Instant Insanity" demonstration in 1971.
45900	A second arm of the same type,  with  only  minor  modifications,  is
46000	currently  being interfaced to the computer.   Both arms will share a
46100	common work space and will  be  able  to  be  used  individually,  or
46200	together for two handed tasks.
46300	
46400	Manipulator  research  moves  ahead  simultaneously  on three fronts:
46500	control, primitives and devices.      Tasks are  translated  into  an
46600	existing  hand  language and where the task cannot be accomplished in
46700	terms of existing primitives, new  primitives  are  formulated  in  a
46800	consistent  manner.     When a new primative is formulated, real time
46900	routines are developed to perform the action.  This  also  leads,  in
47000	many  cases,  to  the  development of new hardware.       Also as new
47100	developments occur in hardware design, existing  real  time  routines
47200	are either re-written or modified.
47300	
47400	Research is now concentrating on the following areas:
47500	
47600	Strain gauge instrumentation of the wrist, in order to allow for more
47700	sensitive interaction of the hand with a  task.     High  resolution,
47800	high  sensitivity  touch sensors, to increase tactile recognition and
47900	control abilities.     These developments will lead in  turn  to  new
48000	control concepts.
48100	
48200	Development  of  tools  for the hand. These tools can be instrumented
48300	such that the computer can feel and or see at the end  of  the  tool.
48400	This  should  be  a very important area of research as it goes beyond
48500	present human capabilities.
48600	
48700	Two handed arm control, the planning and execution of tasks requiring
48800	the  coordinated  motion of both arms. This will require an extension
48900	of the existing servo program and  the  analysis  of  synchronization
49000	problems.   The  development of collision avoiders, so that arms will
49100	not collide with other objects and with each other.
49200	
49300	Modification of  the  existing  arm  design  to  provide  for  better
49400	incremental motions, more degrees of freedom and a more rugged design
49500	suitable for industrial use.   Development of arm  and  eye  computer
49600	interfaces.     These  contain synchronized analog/digital converters
49700	for input and current controlled drivers for output.
49800	
49900	The development of new hands is expected as more  complex  tasks  are
50000	attempted.    At present, with only one hand, we have been performing
50100	the positioning type of hand  actions.     When  the  second  arm  is
50200	operational we will be able to perform holding and pulling actions as
50300	well.
50400	
50500		DEMONSTRATIONS OF MANIPULATION
50600	
50700	Operations controlled by force have been demonstrated  by  turning  a
50800	crank  and  screwing  a  nut  onto  a  bolt.  The  hand  language and
50900	manipulation facilities are shown in the assembly of  a  small  pump.
51000	The  positions of the parts are known approximately; they are located
51100	on a pallet as expected in industrial practice.  By touch,  the  hand
51200	locates  position  and  orientation of the pump body more accurately.
51300	Aligning pins are inserted into the holes to guide the gasket and the
51400	pump  cover.  Screws are obtained from a feeder, as is standard.  All
51500	operations have  been  performed  in  the  assembly,  but  the  total
51600	reliability  is  still  sufficiently  low  that  one error or another
51700	occurs.  Vision has not been used in the assembly.
51800	
51900		SYSTEM REQUIREMENTS FOR ADVANCED AUTOMATION
52000	
52100	A key element in the development of any large programming  effort  is
52200	the  software  support.   There has been continual effort along these
52300	lines at the Stanford Artificial  Intelligence  Laboratory  and  many
52400	useful  ideas have been developed.  We propose to extend the research
52500	along two separate lines:  support for research and  system  software
52600	for  operating  automated systems.  We will briefly describe our past
52700	efforts in this area and what we propose to undertake.
52800	
52900	The Stanford Artificial  Intelligence  Laboratory  has  pioneered  in
53000	several  areas  of  system  programming,  including  display-oriented
53100	timesharing systems, interactive graphics, and programming languages.
53200	
53300	A great deal of additional effort has gone  into  direct  support  of
53400	hand-eye  research.     Very  early, we established a set of standard
53500	formats and library routines [Pingle 72a].  A  programming  language,
53600	SAIL, was implemented in 1968.  The language incorporates special
53700	
53800	11  features  for  large real-world-oriented problem solving systems
53900	and is currently in use at a number of  Artificial  Intelligence  and
54000	other laboratories.
54100	
54200	As  we began to assemble larger and more sophisticated collections of
54300	programs, we felt a need  for  flexible  mechanisms  for  cooperating
54400	sequential  processes.  The resulting monitor and language features [
54500	Feldman & Sproull 1971]  have  worked  well  for  us  and  have  been
54600	influential  on  other  projects.      Recently,  we  have  added  an
54700	additional set of features  to  SAIL  [Feldman  1972]  to  facilitate
54800	non-deterministic   and   event-directed  programming.     The  early
54900	reactions to this work, both within the laboratory  and  without  has
55000	been very encouraging.
55100	
55200	
55300	A major share of the programming research effort will be concentrated
55400	on developing a programming system for automated assembly.  This will
55500	combine ideas  from  our  past  work  on  languages,  representation,
55600	computer  graphics and arm control.  We view this task as a branch of
55700	the larger effort on automatic programming being undertaken  in  this
55800	laboratory aad elsewhere.
55900	
56000	
56100	
56200	
56300	
56400	
56500	
56600	
     

00100	[Anderson 1972] R. Anderson,"Programmable Automation, The Future
00200	of Computers in Manufacturing", Datamation, Dec 1972.
00300	
00400	[Agin] G. Agin, "Description and Representation of  Curved  Objects",
00500	Ph.D.  Thesis, Computer Science Dept., Stanford University, September
00600	1972.
00700	
00800	[Bajcsy 1972] "Computer Identification of Visual Texture" Computer Science
00900	Department, Stanford University, (1972).
01000	
01100	[Baumgart 1972a] B. Baumgart, "GEOMED - A Geometric Editor", May 1972.
01200	SAILON-68,Artificial Intelligence Lab, Stanford University,
01300	
01400	[Baumgart 1972b] Baumgart, Bruce G., WINGED EDGE POLYHEDRON REPRESENTATION, 
01500	AIM 179,Artificial Intelligence Lab, Stanford University,October 1972.
01600	          (forthcoming)
01700	
01800	[Binford 1972]"Sensor Systems for  Manipulation",  Conference  on  Remotely  Manned
01900	Systems, JPL, Pasadena, Calif, Sept 1972.
02000	
02100	[Binford 1971] "Visual  Perception  by  Computer",  Invited  paper  at  IEEE Systems
02200	Science and Cybernetics, Miami, Dec 1971.
02300	
02400	[Binford 1970] T.O.Binford, "Ranging by Laser", Internal Report,
02500	Artificial Intelligence Lab, Stanford University, Oct 1970.
02600	
02700	[Earnest 1967] L.D.Earnest,"Choosing an eye for a Computer";
02800	AI Memo 51, Artificial Intelligence Lab, Stanford University, 1967.
02900	
03000	[Falk 1971] G. Falk, "Scene Analysis Based on Imperfect Edge Data", Proc. 2IJCAI,
03100	Brit. Comp. Soc., Sept. 1971.
03200	
03300	[Feldman 1972] with others,"Recent Developments in SAIL-An Algol Based Language for Artificial 
03400	Intelligence", (with others).  CS 308, Stanford University, August 1972.
03500	To appear in Proc. Fall Joint Computer Conference, 1972.
03600	
03700	[Gill] Gill, Aharon, VISUAL FEEDBACK AND RELATED PROBLEMS IN COMPUTER
03800	         CONTROLLED HAND EYE COORDINATION, AIM  178,October 1972.
03900	
04000	[Grape 1969] "Computer Vision through Sequential Abstraction",
04100	Artificial Intelligence Laboratory, Internal Report, 1969.
04200	
04300	[Grape 1970] "On predicting and Verifying Missing Elements in Line-Drawings"
04400	Artificial Intelligence Laboratory, Internal Report, March 1970.
04500	
04600	[Hueckel] M.H. Hueckel, "An Operator Which Locates Edges in Digitized
04700	Pictures",  Stanford  Artificial  Intelligence  Laboratory   AIM-105,
04800	December 1969, JACM, Vol. 18, No. 1, January 1971.
04900	
05000	[Minsky 1972] "Artificial Intelligence Progress Report", AI Memo 252,
05100	MIT AI Lab, (1972);
05200	
05300	[Paul]  R.   Paul,  "Trajectory  Control  of  a  Computer  Arm",  2nd
05400	International Joint Conference on Artificial Intelligence, 1971.
05500	
05600	[Paul] R. Paul, "Modelling, Trajectory Calculation and Servoing of  a
05700	Computer  Controlled  Arm",  Ph.D.   Thesis,  Computer Science Dept.,
05800	Stanford University, Sept. 1972.
05900	
06000	[Pingle   1972a]   Karl   K.  Pingle,  "Hand/Eye  Library",  Stanford
06100	Artificial Intelligence  Laboratory  Operating  Note  35.1,  January,
06200	1972.
06300	
06400	[Pingle  1972b] K.K. Pingle and J.M. Tenenbaum, "An Accomodating Edge
06500	Follower", IJCAI-2. Proceedings of the 2nd.  International  Symposium
06600	on Industrial Robots, May 1972, Chicago, Illinois.
06700	
06800	[Quam 1972] L. H. Quam, S. Liebes, Jr., R. B. Tucker, M.  J.  Hannah,
06900	B.  G.  Eross,  "Computer  Interactive  Picture Processing", Stanford
07000	Artificial Intelligence Laboratory AIM-166, April 1972.
07100	
07200	[Scheinman] V.  D.  Scheinman,  Design  of  a  Computer  Manipulator,
07300	Stanford Artificial Intelligence Project, Memo AIM-92, June 1969.
07400	
07500	[Sobel] Irwin Sobel, "Camera Models and Machine Perception", Stanford
07600	Artificial Intelligence Project Memo AIM-121, May, 1970.
07700	
07800	[Swinehart] D. Swinehart, R.  Sproull,  "SAIL",  Stanford  Artificial
07900	Intelligence Laboratory Operating Note 57, November 1969.